Risk premiums for conversion-based online advertisement bidding

ABSTRACT

An advertiser specifies a conversion-based bid for a conversion event associated with an ad. If a conversion event occurs for the ad, an effective conversion-based bid can be adjusted by a risk premium associated with the ad. An account associated with the advertiser can be debited based upon the adjusted effective conversion-based bid.

CROSS-REFERENCE TO RELATED APPLICATION

This is a continuation of U.S. application Ser. No. 12/397,524, filedMar. 4, 2009, the disclosure of which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The subject matter of this application is generally related to onlineadvertising.

BACKGROUND

The rise of the Internet has enabled access to a wide variety of contentitems, e.g., video and/or audio files, web pages for particularsubjects, news articles, etc. Such access to these content items haslikewise enabled opportunities for targeted advertising. For example,content items of particular interest to a user can be identified by asearch engine in response to a user query. The query can include one ormore search terms, and the search engine can identify and, optionally,rank the content items based on the search terms in the query andpresent the content items to the user (e.g., according to the rank).This query can also be an indicator of the type of information ofinterest to the user. By comparing the user query to a list of keywordsspecified by an advertiser, it is possible to provide targetedadvertisements to the user.

Another form of online advertising is advertisement syndication, whichallows advertisers to extend their marketing reach by distributingadvertisements to additional partners. For example, third party onlinepublishers can place an advertiser's text or image advertisements on webpages that have content related to the advertisement. Because the usersare likely interested in the particular content on the publisherwebpage, they are also likely to be interested in the product or servicefeatured in the advertisement. Accordingly, such targeted advertisementplacement can help drive online customers to the advertiser's website.

Advertisers can bid for placements based upon how much the advertiservalues the placement. In some examples, the advertiser can bid basedupon impressions of the advertisement. In such examples, the advertiseris charged whenever the advertisement is served. In other examples, theadvertiser can bid based upon a click-through for the advertisement. Insuch examples, the advertiser is charged only when a user clicks on theadvertisement after the advertisement is served to the user. In someexamples, a second price auction can be used to identify a winning bid.In a second price auction, the bidder with the highest bid is identifiedas the winner. The winning bid is defined as a bid that is incrementallymore than the next highest maximum bid. Thus, the winner of the auctionpays slightly more than the next highest maximum bid specified by theuser. The winning bid is the cost paid by the advertiser for theadvertising slot. Thus, the cost to the advertiser is often a discountedvalue of the maximum bid specified by the advertiser (e.g.,Discount=1−Cost/MaxBid).

SUMMARY

An advertiser specifies a maximum conversion based bid (e.g., CPA bid orother target) for a conversion event associated with an ad. Adetermination can be made as to whether the maximum conversion based bidqualifies the advertisement for a placement. If a conversion from theplacement is identified, an effective conversion based bid can beadjusted using a risk premium associated with the online advertisement.The effective conversion-based bid can be derived from the specifiedmaximum conversion based bid for the advertisement. An accountassociated with the advertiser can be debited by the adjusted effectiveconversion-based bid.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an implementation of an online advertisingsystem.

FIG. 2 illustrates an implementation of a user interface for specifyingkeyword bidding options.

FIG. 3 illustrates an implementation of a user interface for settingconversion based bids.

FIG. 4 is a block diagram of an implementation of an advertisingmanagement system for implementing value-based bidding with risk premiumallocation.

FIG. 5 is a flow diagram of an implementation of a risk premiumallocation process using conversion data and ad impression context data.

FIG. 6 is a block diagram of an implementation of an architecture forthe ad management system shown in FIG. 4, which can be configured toimplement the process shown in FIG. 5.

DETAILED DESCRIPTION Advertising System Overview

FIG. 1 is a block diagram of an implementation of an online advertisingsystem 100. In some implementations, one or more advertisers 102 candirectly, or indirectly, enter, maintain, and track advertisement (“ad”)information in an advertising management system 104. The ads may be inthe form of graphical ads, such as banner ads, text only ads, image ads,audio ads, video ads, ads combining one of more of any of suchcomponents, etc. The ads may also include embedded information, such asa links, meta-information, and/or machine executable instructions. Oneor more publishers 106 may submit requests for ads to the system 104.The system 104 responds by sending ads to the requesting publisher 106for placement on one or more of the publisher's web properties (e.g.,websites and other network-distributed content).

Other entities, such as users 108 and the advertisers 102, can provideusage information to the system 104, such as, for example, whether ornot a conversion or click-through related to an ad has occurred. Thisusage information can include measured or observed user behavior relatedto ads that have been served. The system 104 performs financialtransactions, such as crediting the publishers 106 and charging theadvertisers 102 based on the usage information.

A computer network 110, such as a local area network (LAN), wide areanetwork (WAN), the Internet, or a combination thereof, connects theadvertisers 102, the system 104, the publishers 106, and the users 108.

One example of a publisher 106 is a general content server that receivesrequests for content (e.g., articles, discussion threads, music, video,graphics, search results, web page listings, information feeds, etc.),and retrieves the requested content in response to the request. Thecontent server may submit a request for ads to an ad server in thesystem 104. The ad request may include a number of ads desired. The adrequest may also include content request information. This informationcan include the content itself (e.g., page or other content document), acategory corresponding to the content or the content request (e.g.,arts, business, computers, arts-movies, arts-music, etc.), part or allof the content request, content age, content type (e.g., text, graphics,video, audio, mixed media, etc.), geo-location information, etc.

In some implementations, the content server can combine the requestedcontent with one or more of the ads provided by the system 104. Thiscombined content and ads can be sent to the user 108 that requested thecontent for presentation in a viewer (e.g., a browser or other contentdisplay system). The content server can transmit information about theads back to the ad server, including information describing how, when,and/or where the ads are to be rendered (e.g., in HTML or JavaScript™).

Another example publisher 106 is a search service. A search service canreceive queries for search results. In response, the search service canretrieve relevant search results from an index of documents (e.g., froman index of web pages). Search results can include, for example, listsof web page titles, snippets of text extracted from those web pages, andhypertext links to those web pages, and may be grouped into apredetermined number of (e.g., ten) search results.

The search service can submit a request for ads to the system 104. Therequest may include a number of ads desired. This number may depend onthe search results, the amount of screen or page space occupied by thesearch results, the size and shape of the ads, etc. In someimplementations, the number of desired ads will be from one to ten, orfrom three to five. The request for ads may also include the query (asentered or parsed), information based on the query (such as geo-locationinformation, whether the query came from an affiliate and an identifierof such an affiliate), and/or information associated with, or based on,the search results. Such information may include, for example,identifiers related to the search results (e.g., document identifiers or“docIDs”), scores related to the search results (e.g., informationretrieval (“IR”) scores), snippets of text extracted from identifieddocuments (e.g., web pages), full text of identified documents, featurevectors of identified documents, etc. In some implementations, IR scorescan be computed from, for example, dot products of feature vectorscorresponding to a query and a document, page rank scores, and/orcombinations of IR scores and page rank scores, etc.

The search service can combine the search results with one or more ofthe ads provided by the system 104. This combined information can thenforwarded to the user 108 that requested the content. The search resultscan be maintained as distinct from the ads, so as not to confuse theuser between paid advertisements and presumably neutral search results.

Finally, the search service can transmit information about the ad andwhen, where, and/or how the ad was to be rendered back to the system104.

As can be appreciated from the foregoing, the advertising managementsystem 104 can serve publishers 106, such as content servers and searchservices. The system 104 permits serving of ads targeted to documentsserved by content servers. For example, a network or inter-network mayinclude an ad server serving targeted ads in response to requests from asearch service with ad spots for sale. Suppose that the inter-network isthe World Wide Web. The search service crawls much or all of thecontent. Some of this content will include ad spots (also referred to as“inventory”) available. More specifically, one or more content serversmay include one or more documents. Documents may include web pages,email, content, embedded information (e.g., embedded media),meta-information and machine executable instructions, and ad spotsavailable. The ads inserted into ad spots in a document can vary eachtime the document is served or, alternatively, can have a staticassociation with a given document.

In some implementations, the system 104 includes a value-based biddingsystem which predicts or estimates a conversion rate and conversionvalue of a given ad impression for an advertiser 102 using conversiondata and ad impression context, as described in reference to FIGS. 2-5.In further implementations, the system 104 includes a risk premiumadjustment system used to adjust price paid by the advertiser 102 toaccount for the risk associated with misprediction of click-throughand/or conversion rates, or the risk that an advertiser 102 will hideconversions through underreporting or adjustment of conversionparameters.

User Interface Examples

FIG. 2 illustrates an implementation of a user interface 200 forspecifying keyword bidding options. In some implementations, advertiserscan interact with an online ad targeting service (e.g., ad managementsystem 104) through the user interface 200. In this particularimplementation, the user interface 200 includes a campaign managementtab 202 which presents several bidding strategy options that can beselected by the advertiser using a mouse or other input device.

A first option 204 allows the advertiser to specify a maximum monetaryvalue the advertiser is willing to pay for a click or impression. Theadvertiser can choose the first option 204 to control each bid or makefrequent bid adjustments. A second option 206 allows the advertiser toset a 30-day budget and to manage bids to get the most clicks. Theadvertiser can choose the second option 206 to automatically get clickswithin the advertiser's budget. A third option 208 allows the advertiserto set a target bid (e.g., CPA target) for each of the advertiser'skeywords/ad groups. The advertiser can choose the third option 208 toautomatically get the most conversions for the advertiser's target bidand to implement automated value-based bidding. The advertiser canchoose a fourth option 210 to set a conversion-based bid. The advertisercan use the fourth option to pay only for sales, thereby being able tocompare the advertisement cost directly to a sale. As shown in FIG. 2,the user can select an option by first clicking on a bubble or otheruser interface element (e.g., a button), then clicking a “Continue”button representation 212 to receive a new dialog.

Although the user interface 200 allows for specifying a CPA target,other target metrics are possible, such as Return-On-Advertising Spend(ROAS), Return-On-Investment (ROI) and any other appropriate metrics.

FIG. 3 illustrates an implementation of a user interface 200 configuredfor setting conversion-based bids upon the selection of theconversion-based bidding option of FIG. 2 (e.g., fourth option 210). Asimilar interface can be provided upon selection of the target biddingoption in FIG. 2 (e.g., third option 208).

In this particular implementation of the user interface 200, thecampaign management tab 202 presents a new dialog in response to theadvertiser clicking/selecting the third option 208 and clicking the“Continue” button representation 212 in the previous dialog shown inFIG. 2. In some implementations, the new dialog includes two tabs: a“Set Individually” tab 300 for allowing a user to edit ad groupsindividually and an “Edit all in one box” tab 303 for allowing the userto edit all ad groups with a single box. In the example shown, the “EditIndividually” tab 300 is selected by the advertiser. The tab 300 allowsthe advertiser to set a CPA target for each of the advertiser's adgroups. If an ad group has insufficient conversion data available togenerate a meaningful prediction of its conversion rate, then a defaultMax CPC bid specified by the advertiser can be used.

In the example shown, the advertiser has two active ad groups 306:“California Hotels” and “Florida Hotels.” Tab 303 allows the user toinput a single CPA target for all the advertiser's ad groups 306. Theadvertiser can separately specify a different CPA bid for each ad group306 using text boxes 308. When the CPA is specified, the advertiser canclick the Save Changes button representation 310 to save the bids andactivate the campaign. The remainder of the tab 300 is used to presentcampaign data, such as the number of clicks or impressions,click-through-rate (CTR), average CPC, cost, conversion rate, etc.

Of particular interest in the tab 300 is the presentation of arecommended CPA bid 312 for providing the advertiser with guidance inselecting a CPA bid. The recommended CPA bid is theoretically equivalentto the advertiser's current CPC bids. For example, if the advertisercurrently has a CPC bid of $0.30 and a conversion rate of 5%, therecommended CPA bid for conversions would be $6.00 ($6.00*5%=$0.30). Inpractice the advertiser's current CPC bids will typically vary from adto ad and from keyword to keyword for a single ad. In such cases, therecommended CPA bid for a conversion event can be computed using

$\begin{matrix}{{{CPA} = \frac{\sum\limits_{i = 1}^{N}{{Max}\; {CPCbid}_{i}}}{\sum\limits_{i = 1}^{M}{Conversions}_{i}}},} & (1)\end{matrix}$

where the numerator of [1] is the sum of N Max CPC bids over all theclicks that the advertiser received during a relevant period of time(e.g., over the past month), and the denominator of [1] is the totalnumber of conversions M that resulted from these clicks.

It should be noted that the user interfaces 200 shown in FIGS. 2 and 3are merely examples, and other user interfaces having more or fewer userinterface elements, or different user interface elements, can be used toprovide advertisers access to the functionality described herein.

Advertising Management System for Conversion-Based Bidding

FIG. 4 is a block diagram of an implementation of an ad managementsystem 400 for implementing conversion-based bidding. In someimplementations, the system 400 generally includes a conversion manager402, a web server 404, and an ad server 406. The system 400 is operableto communicate with publishers 414, advertiser 416 and users 418, overone or more networks 420 (e.g., the Internet, intranet, Ethernet,wireless network).

In some implementations, a publisher 414 can request an ad from the adserver 406. In response to the request, one or more ads (e.g., imageads) are sent to the publisher 414. The advertisements sent to theadvertisers can be selected based upon an auction (e.g., a second priceauction). In those instances where the bids are based upon multiplebidding paradigms (e.g., CPA, CPC, CPM), the bids can be converted to acommon bidding paradigm and the winning bid can be identified. In thoseimplementations where a CPA bid is included in the auction, the CPAbased bid can be converted to a CPM or CPC based maximum bid. If theconverted bid wins the auction, a winning bid can be identified as a bidthat is incrementally more than the next highest maximum CPM or CPCbased bid.

The ad(s) can be placed on, for example, a web property owned oroperated by the publisher 414 (e.g., a web site). In someimplementations, the web page can have a page content identifier (ID),which can be used by the ad server 406 to determine ad context fortargeting ads that the user 418 will be receptive to. In someimplementations, the ad context can be determined using clusteringtechnology and geographic location data.

In some implementations, when the user 418 clicks an ad served by the adserver 406, the user 418 is directed to a landing page on web property(e.g., a web site) of the advertiser 416. The user 418 may then performa conversion event at the website (e.g., make a purchase, register). Theconversion event generates conversion data which is sent to the system400 and stored in a repository 408 (e.g., MySQL® database). In thismanner, a conversion history can be accumulated and maintained for eachad or ad group in an advertiser's ad campaign.

If the winning bid was a conversion based bid, the winning bid can beconverted to a CPA based winning bid and the advertiser can be chargedthe price associated with the CPA based winning bid. Alternatively, adiscount can be calculated from the converted maximum CPM or CPC basedbid (e.g., Discount=1−(WinningBid/ConvertedMaxBid)). The calculateddiscount can then be applied to the maximum CPA bid to identify the costcharged to the advertiser for the conversion.

Another approach to identifying the cost charged to the CPA based bid isto use an average click auction discount weighted by the predictedconversion rate (e.g., average auction discount over all clicks weightedby predicted conversion rate). One result of computing auction discountin this way is that it equalizes click and conversion costs for anadvertiser by accounting for mispredictions associated with thepredicted conversion rate. In some implementations, the fact that a CPAbased bid wins an auction does not affect a publisher. For example, thepublishers can be paid based upon click-through for an advertisementregardless of whether a conversion occurs and the system can assume therisk for misprediction of the conversion rate.

In some implementations, an advertiser 416 can access the system 400through the network 420 and the web server 404 using, for example, a webbrowser (e.g., Microsoft® Internet Explorer, Mozilla™, Firefox™, or thelike). The web server 404 serves the advertiser 416 one or more webpages presenting a dialog for allowing the advertiser 416 to manage adcampaigns, as described in reference to FIGS. 2 and 3.

In some implementations, the advertiser 416 can use the dialog tospecify a default click-based bid (e.g., maximum CPC or “Max CPC”) and atarget bid (e.g., “CPA target” or “ROAS target”) for each keyword or adgroup in an ad campaign. The default Max CPC can be used to predict aconversion rate for an ad or ad group when there is insufficientconversion data available (e.g., a new ad or ad group). For example, aconversion rate can be estimated by dividing a default Max CPC bid by atarget CPA bid. Alternatively, the default Max CPC can be used insteadof predicting a conversion rate when there is insufficient informationto do so.

In some implementations, the maximum cost per action (CPA) (e.g.,conversion) can be modified by a risk premium allocation subsystem 422to identify a target maximum CPA bid. The risk premium allocationsubsystem 422 can adjust the maximum CPA bid to account for the riskassociated with misprediction of click-through and/or conversion rates,or the risk that an advertiser will hide conversions throughunderreporting or adjustment of conversion parameters. For example, themaximum CPA bid specified by the advertiser can be discounted by thepremium prior to submitting the bid for participation in an auction(e.g., a second-price auction). Such discounting of the maximum CPA bidfacilitates the application of a risk premium to be levied on a winningbid by the risk premium allocation subsystem 422 after a conversion isidentified without exceeding the maximum CPA bid specified by theadvertiser. Using such application of a risk premium, the full amount ofthe risk premium can be recouped. However, such application of the riskpremium can affect the traffic distribution for advertisements, therebyaffecting return on investment.

Moreover, if the CPA bid is discounted by the risk premium, the price ofthe advertisement slot might be reduced, thereby reducing revenue forpublishers and advertising servers, while maintaining the same cost tothe advertiser. For example, in these models there are two cases: 1)when *all* advertisers are paying a premium (Ad Exchange model), and 2)where only a *subset* of advertisers are paying a premium (the case forCO). These cases can be very different. In the first case when you arereducing the bid for all advertisers, there is probably no issue withreduced spending because you are applying a multiplier to all bids inthe auction. In the second case, while computing the auction price(e.g., winning bid) and ranking, one can identify a total opportunitycost for placing the advertisement (which is actual_MaxCPC*pCTR) at aspecific slot, rather than the reduced_MaxCPC*pCTR. This can also showdecreased spending in the auction, because an advertiser not paying thepremium whose bid is now above the advertiser paying the premium nowpays less than the total opportunity cost.

As an example, consider an auction for which four ads (A, B, C, and D)are competing for three slots. Assuming equal pCTR, the ads can beranked by MaxCPC. In this example, ad A has a maximum CPC of 9, ad B hasa maximum CPC of 8, ad C has a maximum CPC of 7.5 and ad D has a maximumCPC of 5. The ranking of the advertisements can be identified, in order,as A, B, C and D, and the total expected revenue for the auction wouldbe the sum of the three lowest bids, or 20.5 (e.g., 8+7.5+5=20.5). Ifthe bid for advertisement B is reduced by 10% based on the risk premium,the new ranking would be A: 9, C: 7.5, B: 7.2 and D: 5. The totalexpected revenue can be calculated as: 7.5+(7.2+0.80)+5=20.5 whichappears to be the same as the expected revenue with no bid modification.However, this assumes that the CTR is the same across all slots. Inpractice this assumption is not true. Higher ordered ad slots have ahigher CTR. So the above case leads to reduction in revenue. Moreover,there is a defined reduction in revenue if the ad due to reduced biddrops out of the auction.

In other implementations, a risk premium can be charged post auction.For example, the risk premium can be charged as part of the auctiondiscount. In such implementations, the fee can be charged such that theCPA does not exceed the maximum CPA bid specified by the advertiser.However, because the amount risk premium is a function of the auctiondiscount, because the cost is capped at the maximum CPA bid, the riskpremium will vary based upon the auction. Such application of the riskpremium would produce no change in the traffic distribution, since themaximum CPA bid is not adjusted prior to conversion to CPC or CPM basedbids. If the average difference between the winning CPA and the maximumCPA is lower than the risk premium, the fees can be recuperated from theadvertiser and the net revenue for the advertising system increases bythe percentage added by the risk premium without change in trafficdistribution. In those instances where the average difference betweenthe winning CPA and the maximum CPA bid is less than the risk premium,some implementations can exclude such advertisers from participation.

In some implementations, the target CPA bid specified by the advertiser416 and as modified by the risk premium allocation subsystem 422 isprovided by the ad server 406 to the conversion manager 402 where it canbe combined with a predicted conversion rate to produce a new oradjusted Max CPC bid. In some implementations, the conversion manager402 includes a learning model 412, which can be built from theconversion data and other information (e.g., how a particular queryconverts across all advertisers). During, for example, an ad auction,the learning model 412 can be used to predict a conversion rate for apotential ad impression. A conversion rate measures how many visits to aweb property “convert” to a sale or “action” as defined by theadvertiser. The conversion rate metric is generally given by

$\begin{matrix}{{{Conversion}\mspace{14mu} {Rate}} = {\frac{\# \mspace{14mu} {of}\mspace{14mu} {sales}\mspace{14mu} {from}\mspace{14mu} a{\mspace{11mu} \;}{given}\mspace{14mu} {ad}}{\# \mspace{14mu} {of}\mspace{14mu} {visits}\mspace{14mu} {to}\mspace{14mu} {web}\mspace{14mu} {property}\mspace{14mu} {from}\mspace{14mu} {ad}}.}} & (2)\end{matrix}$

In some implementations, the learning model 412 is a machine learningsystem model that includes rules for mapping impression context featuresto conversion rate predictions. For example, the learning model 412 mayinclude, but is not limited to, the following rules:

ad appears on America Online® (AOL): probability multiplier=1.1

user is from USA: probability multiplier=0.85

user is from UK: probability multiplier=0.95

Time of day 9 am-noon: probability multiplier=0.9

Using these rules, if an ad is shown on AOL to a user from the UnitedKingdom (UK) at 10:00 AM, then the default conversion rate from [2]would be multiplied by the probability multipliers 1.1, 0.95 and 0.9,corresponding to the impression context features AOL, UK and time ofday, respectively. Thus, if the default conversion rate is 0.2, then theconversion rate prediction for ads satisfying the rules would be(0.2*1.1*0.95*0.9)=18.81%.

Similarly, if the same ad is shown on AOL to a user in the United Statesat the same time, then the default conversion rate computed in [2] wouldbe multiplied by the probability multipliers 1.1, 0.85 and 0.9,corresponding to the impression context features: AOL, USA and time ofday, respectively. Thus, if the default conversion rate for the ad is0.2, then the conversion rate prediction for ads satisfying the ruleswould be (0.2*1.1*0.85*0.9)˜16.8%.

In some implementations, the rules used by the learning model 412 can bespecific to a particular ad. For example, the learning model 412 mayinclude the following rules for a fictitious “ad 17354” for a Londonrestaurant:

showing ad 17354 to a user from USA: probability multiplier=0.5

showing ad 17354 to a user from UK: probability multiplier=3.0.

In the examples shown, the computation could also be performed in termsof odds rather than probabilities since odds have better behavedmathematical properties. For example, with odds one can avoid problemsassociated with a probability greater than one.

Techniques for deriving rules from conversion data using machinelearning systems is described in U.S. patent application Ser. No.10/712,263, for “Targeting Advertisements Based on Predicted Relevanceof the Advertisements,” and U.S. patent application Ser. No. 11/321,046,for “Predicting Ad Quality.”

In some implementations, the predicted conversion rate (pCVR) can beused to compute or adjust an advertiser's click-based bid (e.g., Max CPCbid). For example, if the CPA specified by an advertiser is $50 and thepredicted conversion rate is 2%, the Max CPC can be automaticallyadjusted to $1 using the formula

MaxCPC(adjusted)=MaxCPA*pCVR.  (3)

If there is insufficient conversion data available to compute pCVR, thenthe advertiser's specified default Max CPC can be used as a metric,until sufficient conversion data has been gathered for the ad, at whichtime [3] can be used to automatically compute or adjust the Max CPC.During the course of an ad campaign, the conversion data (and optionallythe learning model) can gradually change over time as more data isaccumulated, while the impression context varies from auction toauction. These changes can result in the calculation of a new predictedconversion rate. The new predicted conversion rate can then be used in[3] “on-the-fly,” so that the advertiser's default Max CPC bid used inthe auction is automatically and continuously computed or adjustedduring the ad campaign or auction.

Equation [3] is one example of a formula calculating an adjusted MaxCPC. Other formulas are possible that combine a default CPC andconfidence information. For example, let V be the number of conversionsobserved for an ad or ad group (i.e., the amount of conversion dataavailable), and X1 and X2 be threshold values, where X1<X2. Then,

for V<=X1,

MaxCPC=DefaultCPC;

for X1<V<X2,

MaxCPC=CPA*pCVR*Z+DefaultCPC*(1−Z), where Z=(V−X1)/(X2−X1);

and

for V>=X2,

MaxCPC=CPA*pCVR.

Based on the above equations, a default CPC can be used when V is lessthan or equal to X1, equation [3] can be used when V is greater than orequal to X2, and a linear blend of the two when V is between X1 and X2.Note that plugging V=X1 or V=X2 into the second formula producesDefaultCPC and CPA*pCVR, respectively.

Equation [3] assumes that all conversions for the ad or ad group are thesame. If the advertiser does not want to specify a single CPA for allconversions associated with an ad or ad group, then in someimplementations, the advertiser can instead specify a target ROAS valuefor each ad or ad group. In that case, the advertiser should also reportback data, such as the value of a conversion event that occurs on itsweb site as part of the historical conversion data. For example, thevalue of a conversion can be the dollar amount of the sale of theadvertised item (e.g., 199.0 for a $199.00 iPod®).

In some implementations, when a user specifies a maximum ROAS bid basedupon conversions, the risk premium allocation subsystem 422 can discountthe maximum ROAS bid using a risk premium. The risk premium can beapplied to adjust the maximum ROAS bid to account for the riskassociated with misprediction of click-through and/or conversion rates,or the risk that an advertiser will hide conversions throughunderreporting or adjustment of conversion parameters. For example, themaximum ROAS bid specified by the advertiser can be discounted by therisk premium prior to submitting the bid for participation in an auction(e.g., a second-price auction) or conversion to a CPC or CPM based bidfor participation in CPC or CPM based auctions, respectively. Suchdiscounting of the maximum ROAS bid facilitates the application of arisk premium to be levied on a winning bid by the risk premiumallocation subsystem 422 after a conversion is identified withoutexceeding the maximum CPA bid specified by the advertiser.

In some implementations, the learning model 412 can predict an expectedconversion value (in addition to a predicted conversion rate) and usethe predicted conversion value to generate an impression-specific CPCbid. For example, assume the keywords “computer” and “computeraccessories” have a conversion rate of 5%, and the average dollar valueof a computer sold is $1000 and the average value of a computeraccessory sold is $100. If an advertiser has the same profit margins onboth items, the advertiser may be willing to pay more to advertise onthe keyword “computer” than the keyword “computer accessories.” In thisexample, conversion value is given by

$\begin{matrix}{{{Conversion}\mspace{14mu} {Value}} = {\frac{\$ \mspace{14mu} {value}\mspace{14mu} {of}\mspace{14mu} {sales}\mspace{14mu} {from}\mspace{14mu} a{\mspace{11mu} \;}{given}\mspace{14mu} {ad}}{\# {\; \mspace{11mu}}{times}\mspace{14mu} {the}\mspace{14mu} {ad}\mspace{14mu} {was}\mspace{14mu} {clicked}}.}} & (4)\end{matrix}$

A ROAS value indicates how many dollars in sales the advertiser wants togenerate for each dollar spent on advertising. In this scenario, thetotal sum of N conversion values, CV_(i), can be divided by the numberof conversions N and the ROAS to get the MaxCPA:

Using [5], if the

$\begin{matrix}{{{Max}\; {CPA}} = {\frac{\sum\limits_{i = 1}^{N}{CV}_{i}}{N*{ROAS}}.}} & (5)\end{matrix}$

advertiser earned $3000 on 50 conversions, and has specified a targetROAS bid of $10, the CPA will be $3000/50/$10=$6.00. Thus, if theadvertiser sold $3000 worth of product in 50 transactions, the averagesale amount is $60. Assuming a ROAS target of $10, then the advertiseris willing to pay an average of $6 in advertising costs for eachconversion. The CPA computed using [5] can then be multiplied by thepredicted conversion rate computed using [3] to compute or adjust theadvertiser's Max CPC bid.

The example above simplifies and assumes that all conversions have thesame value. In some implementations, the learning model 412 is a machinelearning system model that includes rules for mapping impression contextfeatures to conversion value predictions. For example, the learningmodel 412 may include, but is not limited to, the following rules:

ad appears on a web site in the “travel” category: value multiplier=1.1;

ad position is “below the fold”: value multiplier=0.7;

user is located in San Francisco: value multiplier=0.95; and

day of the week is Monday: value multiplier=1.2.

Using these rules, if an ad is shown on a travel site at the bottom ofthe page to a user from San Francisco on a Monday, then the defaultconversion value would be multiplied by the value multipliers 1.1, 0.7,0.95 and 1.2. Thus, if the default conversion value is $18, then theconversion value prediction for ads satisfying the rules would be($18*1.1*0.7*0.95*1.2)=$15.80.

In some implementations, the predicted conversion value (pCVV) can beused to compute or adjust an advertiser's click-based bid (e.g., Max CPCbid). For example, if the ROAS specified by an advertiser is $10 and thepredicted conversion value is $15.80, the Max CPC can be automaticallyadjusted to be $1.58 using the formula

MaxCPA=pCVV/ROAS.  (6)

ROAS can be used directly without CPA. For example, an online travelstore could find that people buy more expensive tickets at night (7-11pm) than in the morning (9 am-noon).

Thus, an advertiser's Max CPC bid for each ad in an auction can bemodified dynamically (i.e., on the fly) for each impression context,providing an advantage over conventional systems that only allowadvertiser's to make a single, static Max CPC bid for each keyword.

In some implementations, the predicted conversion rate can be used torank an ad in an auction. A Max Cost-Per-Mille (CPM) can be computedfrom an expected clickthrough rate (CTR), CPA bid and predictedconversion rate pCVR. The expected CPM can be used to determine winningbids in an ad auction that ranks ad effectiveness using a suitablemetric. For example, the performance of an ad can be measured by aneffective cost of one thousand impressions (eCPM) of the ad. That is,the performance of an ad can be measured by the amount of revenuegenerated by presenting the ad to users one thousand times. The eCPM maybe estimated by multiplying the CPA target, the predicted conversionrate, pCVR, and the predicted CTR, pCTR, for that action multiplied byone thousand:

eCPM=CPA*pCVR*pCTR*1000.  (7)

Machine Learning System Models

Training a machine learning system to accurately predict conversionrates can be difficult due to unreported conversions, which could leadto deflated predictions, conversion latency (e.g., a conversion mayoccur several weeks after the corresponding click) and the need for aconfidence score to translate a conversion rate into a bid.

In some implementations, these effects can be mitigated by training twomachine learning models in parallel. A first model (“Model A”) wouldestimate the average conversion rate for each ad group, while a secondmode (“Model P”) would predict conversion rates for specific adimpressions. The ratio of these two predictions can be used to adjustthe ad group's Max CPC, using the formula

MaxCPC(adjusted)=(default MaxCPC)*P/A,  (8)

where P and A are the conversion rate predictions for Model A and ModelP, respectively. The ratio P/A will cause any bias that affects bothmodels equally to cancel out.

Normalizing Conversion Events

Because conversions are defined by different advertisers and conversionrates of different ads can differ dramatically, a few ad groups withhigh conversion rates can dominate the statistics. For example, considertwo advertisers, A1 and A2, showing the same ad on two web properties,P1 and P2, respectively, with the clicks and conversions shown in TableI. Advertiser A1 defines every click as a conversion and advertiser A2defines a conversion as a purchase after each click.

TABLE I Example Clicks & Conversion for Advertisers A & B P1 P2 Ad TotalA1 1000 100 1100 conversions/1000 conversions/100 conversions/1100clicks = 100% clicks = 100% clicks = 100% A2 1 conversion/100 10 11clicks = 1% conversions/1000 conversions/1100 clicks = 1% clicks = 1%Property 1001 110 Total conversions/1100 conversions/1100 clicks = 90%clicks = 10%As shown in Table I, the result is that the overall conversion rate forP1 is much higher than the overall conversion rate for P2, although theonly difference between the properties is which ads are shown. This canmake it difficult for the learning model to determine what effect theproperty itself has on conversion rates. The differences in thedefinitions of a conversion can be equalized by normalizing theconversions.

In some implementations, a process for designing conversion models usingnormalization and impression context features is described as follows:

1. Identify a set of impression context features that are likely toaffect ad conversion rate.

2. Build a first model for predicting an ad's base conversion rate usingonly ad-related features identified in step 1 (e.g., the number ofclicks and conversions as shown in Table I). The ad's base conversionrate can then be used to normalize a conversion rate generated by asecond, more detailed, model, as described in step 4.

3. Build a second model for predicting the ad's conversion rate usingad-related and non-ad-related features identified in step 1.

4. When training the model in step 3, normalize each conversion by thead's base conversion rate predicted by the first model, so that eachconversion counts as w conversions, where w=x %/base_conversion_rate.”Note that normalizing the conversion rate removes the effect ofdifferent conversion definitions by equalizing the effective conversionrate of all ad groups to x % (e.g., 1%).

5. Denormalize the prediction returned by the model by multiplying it bybase_conversion_rate/x %. Since the model will produce an averageconversion rate of x % for each ad group, the prediction should bemultiplied by base_conversion_rate/x % to produce the correctprediction.

In some cases it may be difficult to extract an ad's base conversionrate from its clicks and conversions. In such cases, the ad's baseconversion rate can be approximated with the ad's average conversionrate.

Using Statistics to Predict Conversion Rates

In some implementations, a statistics-based approach can be employed topredict conversion rates. In this approach, a machine learning systemcan be used to collect the number of clicks and conversions for eachimpression context feature of interest. Statistics (e.g., averages) canthen be calculated from these numbers for use in predicting a conversionrate.

Post Auction and Conversion Processing

After performing the auction using the target bid (e.g., maximum CPC,maximum CPM, etc.), the advertising server 406 can detect whether aconversion has been made. In some implementations, a conversion can bedetected based upon the insertion of one or more code snippets into anadvertiser website, web pages or landing page(s). The inserted one ormore snippets of code can detect the completion of a transaction and cancommunicate the transaction to the ad server 406 and/or conversionmanager 402. In other implementations, the conversion can be reported bythe advertiser. In various implementations, the risk premium for aconversion can be applied pre-conversion or post-conversion.

Upon receipt of notification that a conversion has been detected, therisk premium allocation subsystem 422 can convert an impression or clickbased winning bid associated with the conversion to a CPA based winningbid. In those examples where the winning bid is an impression basedwinning bid (CPM), the impression based winning bid can be divided by anestimated click-through rate to identify an estimated CPC based winningbid. The estimated CPC based winning bid can then be divided by anestimated conversion rate to identify the estimated CPA based winningbid. In those examples where the auction is a CPC based auction, the CPCbased winning bid can be divided by an estimated conversion rate toidentify the estimated CPA based winning bid.

In some implementations, the risk premium allocation subsystem 422 canmultiply the estimated CPA based winning bid by a risk premium toidentify a charged cost to the advertiser for the winning bid. In thoseimplementations where the risk premium allocation subsystem 422discounts the maximum CPA based bid specified by the advertiser by therisk premium prior to submission of the target maximum bid to theauction, a chance that the cost to the advertiser of the conversionexceeds the maximum CPA bid specified by the advertiser is minimized.For example, a CPA based bid of $5.00 can be discounted to account for arisk premium of 20% (e.g., target bid=$5.00/(1+0.2)=$4.17). Thus, if theestimated CPA based winning bid is the target bid, the risk premium canbe applied without exceeding the maximum CPA based bid specified by theadvertiser (e.g., advertiser cost=$4.17*(1+0.2)=$5.00). In suchimplementations, the advertiser is allowed to participate in the auctionwhile bearing the full cost of underreporting risk and miscalculation ofCTR or conversion rate.

In some implementations, the target maximum bid can be set to be themaximum CPA based bid specified by the advertiser. In suchimplementations, the risk premium allocation subsystem 422 can apply arisk premium to the estimated CPA based winning bid up to the maximumCPA based bid specified by the advertiser. For example, if the estimatedCPA based winning bid is $5.00 and the maximum CPA based bid specifiedby the advertiser is $5.50, with a risk premium of 20%, the amountcharged to the advertiser is capped at $5.50 even though the riskpremium would have charged $6.00 to the advertiser in the absence of the$5.50 maximum CPA based bid specified by the advertiser. Thus, a riskpremium is charged while maintaining the ability of the advertiser toparticipate in the auction up to the specified maximum CPA based bid.

In other implementations, the risk premium allocation subsystem 422 canallocate the risk premium by charging a fixed fee or percentage on topof the estimated CPA winning bid. For example, the risk premiumallocation subsystem 422 can charge a fee of $1 per conversion.Alternatively, the risk premium allocation subsystem 422 can charge afee of 15% of the estimated CPA winning bid to the advertiser.

In still other implementations, the risk premium allocation subsystem422 can allocate the risk premium by charging a subscription fee to theadvertiser for using the CPA based bidding system. For example, anadvertiser might sign up for a service whereby the advertiser pays $50per month for up to 30 conversion from the CPA based bidding system.

Risk Premium Allocation Process

FIG. 5 is a flow diagram of an implementation of a risk premiumallocation process 500 using conversion data and ad impression contextdata. In some implementations, the process 500 begins by determiningwhether a target conversion based bid (e.g., CPA target, ROAS target)qualifies the advertisement for placement (510). The target conversionbased bid can be derived from a maximum conversion based bid (e.g.,maximum CPA bid) received from an advertiser.

In some implementations, the target conversion based bid can be derivedby discounting the maximum conversion based bid by a risk premium. Thedetermination of whether a target conversion based bid qualifies theadvertisement for placement can be based upon submission of the targetconversion based bid to a placement auction. In some implementations,submission of the target conversion based bid to a placement auction isfacilitated by conversion of the target conversion based bid to amaximum target click based bid based upon the conversion rate for theadvertisement, or maximum target impression based bid based upon theconversion rate of the advertisement and a predicted click through ratefor the advertisement.

In other implementations, the target conversion based bid is identifiedas the maximum conversion based bid. The maximum conversion based bidcan be converted to a maximum CPC or CPM based bid based upon thebidding paradigm used for the auction.

The process 500 continues by determining whether or not a conversionassociated with the advertisement has occurred (520). The determinationcan be made, for example, based upon receiving feedback from the usercomputer or the advertiser computer when a transaction is made. Suchfeedback can be provided upon execution of one or more program codesnippets causing the advertiser or user to notify an ad server when aconversion occurs. If a conversion has not occurred as a result of theplacement, the process 500 returns to wait for a determination that theadvertisement is eligible for another placement.

If a conversion occurs (520), an effective conversion-based bid can beadjusted based upon a risk premium (530). In those auctions using animpression based bidding paradigm, the effective conversion-based bidcan be identified by dividing the winning impression based bid by anestimated CTR and an estimated conversion rate. In those auctions usinga click based bidding paradigm, the effective conversion-based bid canbe identified by multiplying the winning click based bid by an estimatedconversion rate for the advertisement. In various implementations, therisk premium can be a fixed percentage of the effective conversion basedbid, a fixed fee for each conversion, a subscription fee allocated amongconversions. Other risk premiums can be used. In some implementations,the fee charged to the advertiser can be capped by the maximumconversion based bid specified by the advertiser. Thus, for example, ifthe advertiser specified a maximum conversion based bid of $10.00, andthe effective conversion based bid was $9.50, with a risk premium of10%, the risk premium of $0.95 would increase the cost to the advertiserover $10.00. In such examples, the risk premium can be capped at $0.50(e.g., $10.00-$9.50=$0.50) to ensure that the cost does not exceed themaximum conversion based bid specified by the advertiser.

The process 500 continues by debiting an account associated with theadvertiser based upon the adjusted effective conversion-based bid. Invarious implementations, the account can be a pre-paid account fromwhich funds for the conversion are impounded, or can be a credit basedaccount against which charges are applied and upon which the advertiserperiodically makes payment to bring the account up-to-date. Other typesof accounting or monetary allocation plans can be used.

The foregoing process 500 is one implementation of a risk premiumallocation process. Other processes are possible, including processeswith more or fewer steps. The steps of process 500 need not be performedserially in the order shown. The process 500 can be divided intomultiple processing threads of one or more processor cores and/orparallel processors.

Advertising Management System Architecture

FIG. 6 is a block diagram of an exemplary architecture 600 for the admanagement system 400 shown in FIG. 4, which can be configured toimplement the process 500 shown in FIG. 5.

In some implementations, the architecture 600 includes one or moreprocessors 602 (e.g., dual-core Intel® Xeon® Processors), one or morerepositories 604, 609, one or more network interfaces 608, an optionaladministrative computer 606 and one or more computer-readable mediums610 (e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory,etc.). These components can exchange communications and data over one ormore communication channels 612, which can include various known networkdevices (e.g., routers, hubs, gateways, buses) and software (e.g.,middleware) for facilitating the transfer of data and control signalsbetween devices.

The term “computer-readable medium” refers to any medium thatparticipates in providing instructions to a processor 602 for execution,including without limitation, non-volatile media (e.g., optical ormagnetic disks), volatile media (e.g., memory) and transmission media.Transmission media includes, without limitation, coaxial cables, copperwire and fiber optics. Transmission media can also take the form ofacoustic, light or radio frequency waves.

The computer-readable medium 610 further includes an operating system614 (e.g., Linux server, Mac OS® server, Windows® NT server), a networkcommunication module 616, an advertising management module 618 and apayment system 628.

The operating system 614 can be multi-user, multiprocessing,multitasking, multithreading, real-time and the like. The operatingsystem 614 performs basic tasks, including but not limited to:recognizing input from and providing output to the administratorcomputer 606; keeping track of files and directories oncomputer-readable mediums 610 (e.g., memory or a storage device);controlling peripheral devices (e.g., repositories 604 and 609); andmanaging traffic on the one or more communication channels 612.

The network communications module 616 includes various components forestablishing and maintaining network connections (e.g., software forimplementing communication protocols, such as TCP/IP, HTTP, Ethernet,etc.).

The advertising management module 618 includes an ad server 620, a webserver 622 and a conversion manager 624. The conversion manager 624further includes a learning model 626. The ad server 620 can be a serverprocess or dedicated machine that is responsible for serving ads topublisher web properties and for tracking various information related tothe ad placement (e.g., cookies, user URLs, page content, geographicinformation). The ad server can also include a risk premium allocationsubsystem 621 operable to allocate a risk premium for advertisementsusing conversion based bidding for auction processes. The web server 622(e.g., Apache web page server) serves web pages to advertisers andpublishers and provides a means for advertisers and publishers tospecify a target cost-per-action for use by the conversion manager 624and its learning model 626 to dynamically compute or adjust anadvertiser's click-based bid (e.g., Max CPC bid) or other performancemetric, as described in reference to FIGS. 4 and 5.

The ad repository 604 can include various ads including, withoutlimitation, image ads, text links, video and any other content that canbe placed on a publisher web page and interacted with to drive users toadvertiser properties.

The repository 609 can be used to store conversion data associated withan ad or ad group. The conversion data is used by the conversion manager624 to generate a predicted conversion rate for given ad or ad group, asdescribed in reference to FIGS. 4 and 5.

The payment system 628 is responsible for implementing a paymentprocess, whereby advertisers pay publishers, such as is done in Google'sAdSense™ service. The payment process can be fully or partiallyautomated, and can include human intervention at one or more points inthe payment process.

The disclosed embodiments can be implemented in a computing system thatincludes a back-end component, e.g., as a data server, or that includesa middleware component, e.g., an application server, or that includes afront-end component, e.g., a client computer having a graphical userinterface or a web browser through which a user can interact with animplementation of what is disclosed here, or any combination of one ormore such back-end, middleware, or front-end components. The componentsof the system can be interconnected by any form or medium of digitaldata communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of what being claims or of whatmay be claimed, but rather as descriptions of features specific toparticular embodiments. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable sub-combination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understand as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

A number of embodiments of the invention have been described.Nevertheless, it will be understood that various modifications may bemade without departing from the spirit and scope of the invention.

What is claimed is:
 1. (canceled)
 2. A computer-implemented methodcomprising: determining at an advertisement management system that aspecified conversion-based bid associated with an first onlineadvertisement specified by an advertiser qualifies the first onlineadvertisement for placement by the advertisement management system on aweb property of a publisher, comprising: determining a risk premium tobe charged to the advertiser; discounting the specified conversion-basedbid based on the risk premium to generate a discounted specifiedconversion-based bid; converting the discounted specifiedconversion-based bid to an impression-based bid, and performing animpression-based auction that includes the impression-based bid and oneor more other impression-based bids; providing the first onlineadvertisement for placement on the web property of the publisher;crediting the publisher for the placement of the first onlineadvertisement on the web property; after the advertisement has beenprovided for placement on the web property and after the publisher hasbeen credited for the placement, detecting an occurrence of a conversionevent for the first online advertisement; and in response to detectingthe occurrence of the conversion event, debiting an account associatedwith the advertiser with the advertisement management system for theconversion, wherein debiting the account comprises: determining that thefirst online advertisement was qualified for placement by converting thediscounted conversion-based bid to an impression-based bid and includingthe impression-based bid in an ad auction with the otherimpression-based bids, in response, adjusting a price to be charged tothe advertiser for the conversion based on the risk premium, anddebiting the account associated with the advertiser based on theadjusted price.
 3. The computer-implemented method of claim 2, whereinthe risk premium based at least in part on a risk assumed by theadvertisement management system for crediting the publisher forplacement of the first online advertisement while waiting to charge theadvertiser for conversion events for the first online advertisement. 4.The computer-implemented method of claim 2, further comprising:detecting an occurrence of a conversion event for a second onlineadvertisement associated with a second advertiser; and in response todetecting the occurrence of the conversion event for the second onlineadvertisement associated with the second advertiser: determining thatthe second online advertisement was not qualified for placement byconverting a conversion-based bid to an impression-based bid andincluding the impression-based bid in an ad auction with the otherimpression-based bids, and in response to determining that the secondonline advertisement was not qualified for placement by converting aconversion-based bid to an impression-based bid and including theimpression-based bid in an ad auction with the other impression-basedbids, debiting an account associated with the second advertiser based ona price that is not adjusted based on the risk premium.
 5. The method ofclaim 2, wherein converting the discounted specified conversion-basedbid to the impression-based bid comprises converting the discountedspecified conversion-based bid using a predicted conversion rate and apredicted click-through rate associated with the auction.
 6. The methodof claim 5, further comprising automatically mapping one or moreimpression context features to the predicted conversion rate using alearning model.
 7. The method of claim 6, wherein the learning model isa machine learning system model that includes rules for mapping one ormore impression context features to a predicted conversion rate usingconversion data.
 8. The method of claim 5, further comprising:normalizing the predicted conversion rate to remove an effect ofdifferent conversion definitions.
 9. The method of claim 2, wherein therisk premium comprises a percentage of the specified conversion-basedbid.
 10. The method of claim 2, wherein the risk premium comprises afixed fee.
 11. A system comprising one or more computers and one or morestorage devices storing instructions that when executed by the one ormore computers cause the one or more computers to perform operationscomprising: determining at an advertisement management system that aspecified conversion-based bid associated with an first onlineadvertisement specified by an advertiser qualifies the first onlineadvertisement for placement by the advertisement management system on aweb property of a publisher, comprising: determining a risk premium tobe charged to the advertiser; discounting the specified conversion-basedbid based on the risk premium to generate a discounted specifiedconversion-based bid; converting the discounted specifiedconversion-based bid to an impression-based bid, and performing animpression-based auction that includes the impression-based bid and oneor more other impression-based bids; providing the first onlineadvertisement for placement on the web property of the publisher;crediting the publisher for the placement of the first onlineadvertisement on the web property; after the advertisement has beenprovided for placement on the web property and after the publisher hasbeen credited for the placement, detecting an occurrence of a conversionevent for the first online advertisement; and in response to detectingthe occurrence of the conversion event, debiting an account associatedwith the advertiser with the advertisement management system for theconversion, wherein debiting the account comprises: determining that thefirst online advertisement was qualified for placement by converting thediscounted conversion-based bid to an impression-based bid and includingthe impression-based bid in an ad auction with the otherimpression-based bids, in response, adjusting a price to be charged tothe advertiser for the conversion based on the risk premium, anddebiting the account associated with the advertiser based on theadjusted price.
 12. The system of claim 11, wherein the risk premiumbased at least in part on a risk assumed by the advertisement managementsystem for crediting the publisher for placement of the first onlineadvertisement while waiting to charge the advertiser for conversionevents for the first online advertisement.
 13. The system of claim 11,the operations further comprising: detecting an occurrence of aconversion event for a second online advertisement associated with asecond advertiser; and in response to detecting the occurrence of theconversion event for the second online advertisement associated with thesecond advertiser: determining that the second online advertisement wasnot qualified for placement by converting a conversion-based bid to animpression-based bid and including the impression-based bid in an adauction with the other impression-based bids, and in response todetermining that the second online advertisement was not qualified forplacement by converting a conversion-based bid to an impression-basedbid and including the impression-based bid in an ad auction with theother impression-based bids, debiting an account associated with thesecond advertiser based on a price that is not adjusted based on therisk premium.
 14. The system of claim 11, wherein converting thediscounted specified conversion-based bid to the impression-based bidcomprises converting the discounted specified conversion-based bid usinga predicted conversion rate and a predicted click-through rateassociated with the auction.
 15. The system of claim 14, the operationsfurther comprising automatically mapping one or more impression contextfeatures to the predicted conversion rate using a learning model. 16.The system of claim 15, wherein the learning model is a machine learningsystem model that includes rules for mapping one or more impressioncontext features to a predicted conversion rate using conversion data.17. The system of claim 14, the operations further comprising:normalizing the predicted conversion rate to remove an effect ofdifferent conversion definitions.
 18. The system of claim 11, whereinthe risk premium comprises a percentage of the specifiedconversion-based bid.
 19. The system of claim 11, wherein the riskpremium comprises a fixed fee.
 20. A computer program product encoded onone or more non-transitory computer storage media, the computer programproduct comprising instructions that when executed by one or morecomputers cause the one or more computers to perform operationscomprising: determining at an advertisement management system that aspecified conversion-based bid associated with an first onlineadvertisement specified by an advertiser qualifies the first onlineadvertisement for placement by the advertisement management system on aweb property of a publisher, comprising: determining a risk premium tobe charged to the advertiser; discounting the specified conversion-basedbid based on the risk premium to generate a discounted specifiedconversion-based bid; converting the discounted specifiedconversion-based bid to an impression-based bid, and performing animpression-based auction that includes the impression-based bid and oneor more other impression-based bids; providing the first onlineadvertisement for placement on the web property of the publisher;crediting the publisher for the placement of the first onlineadvertisement on the web property; after the advertisement has beenprovided for placement on the web property and after the publisher hasbeen credited for the placement, detecting an occurrence of a conversionevent for the first online advertisement; and in response to detectingthe occurrence of the conversion event, debiting an account associatedwith the advertiser with the advertisement management system for theconversion, wherein debiting the account comprises: determining that thefirst online advertisement was qualified for placement by converting thediscounted conversion-based bid to an impression-based bid and includingthe impression-based bid in an ad auction with the otherimpression-based bids, in response, adjusting a price to be charged tothe advertiser for the conversion based on the risk premium, anddebiting the account associated with the advertiser based on theadjusted price.
 21. The computer program product of claim 20, whereinthe risk premium based at least in part on a risk assumed by theadvertisement management system for crediting the publisher forplacement of the first online advertisement while waiting to charge theadvertiser for conversion events for the first online advertisement.